Hi! I am a research scientist at Shanghai AI Lab. I got my PhD degree from the University of Hong Kong at the end of 2021, affiliated with the HKU database group and NLP group. I am advised by Prof. Ben Kao. I am also working closely with Lingpeng Kong. Before that, I received my B.E. degree from Wuhan University in 2017.
My research centers around large language models (LLMs) with a special focus on building the next generation of natural language interfaces that can interact with and learn from real-world environments. You can find a prototype of my ambitious goal at OS-Copilot.
I have multiple internship positions available (Language agent, Self-evolving, RL) , please feel free to hit me up with your CV or questions if interested.
🔥 News
- 2024.05: SeeClick and Symbol-LLM are accepted to ACL main conference! See you in Bangkok!
- 2024.04: 🎉 New homepage!
📝 Selected Publications
Arxiv
OS-ATLAS: A Foundation Action Model For Generalist GUI Agents
Zhiyong Wu, Zhenyu Wu, Fangzhi Xu, Yian Wang*, Qiushi Sun, Chengyou Jia, Kanzhi Cheng, Zichen Ding, Liheng Chen, Paul Pu Liang, Yu Qiao.
- Check demos at Our Website
- SOTA GUI grounding and action model upon which you can easily build your own agent. Code
.
- Repost and like us on Twitter
LLMAgents@ICLR 2024
OS-Copilot: Towards Generalist Computer Agents with Self-Improvement
Zhiyong Wu, Chengcheng Han*, Zichen Ding, Zhenmin Weng, Zhoumianze Liu, Shunyu Yao, Tao Yu, Lingpeng Kong.
- Check demos at Our Website
- Build your personal agents at Code
.
- Join our Discord to have fun, or follow us on Twitter
arXiv 2024AgentStore: Scalable Integration of Heterogeneous Agents As Specialized Generalist Computer Assistant, Chengyou Jia, Minnan Luo, Zhuohang Dang, Qiushi Sun, Fangzhi Xu, Junlin Hu, Tianbao Xie, Zhiyong Wu.arXiv 2024Interative Evolution: A Neural-symbolic Self-Training Framework for Large Language Models, Fangzhi Xu, Qiushi Sun, Kanzhi Cheng, Jun Liu, Yu Qiao, Zhiyong Wu.ACL 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents, Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, Zhiyong Wu.ACL 2023Self-adaptive In-context Learning, Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, Lingpeng Kong.ICLR 2023DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models, Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, Lingpeng KongACL 2020Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT, Zhiyong Wu, Yun Chen, et al.
🤖 Interns
- Jiacheng Ye (2022.1-2023.5) EMNLP’22a EMNLP’22b ICML’23
- Sijie Cheng (2022.3-2022.8) AAAI’23
- Yaoxiang Wang (2022.10-2024.4) ACL’23a ACL’23b
- Zhenyu Wu (2022.10-) ACL’23b Under review at NeurIPS’24
- Siyu Ren (2023.8-2024.2) ICLR’24
- Qiushi Sun (2023.6-) COLM’24, Survey Paper
- Fangzhi Xu (2023.8-) ACL’24 Under review at ARR
- Kanzhi Cheng (2023.8-2024.4) ACL’24
- Yi Lu (2023.10-2024.4) Under review at ICLR’25
- Chengyou Jia (2024.3-2024.9) Under review at ICLR’25
📖 Educations
- 2017.09 - 2021.11, PhD, University of Hong Kong.
- 2013.09 - 2017.06, Undergraduate, Wuhan Univeristy.
📚 Full Publication List
🤖 Agents
arXiv 2024OS-ATLAS: A Foundation Action Model For Generalist GUI Agents, Zhiyong Wu, Zhenyu Wu, Fangzhi Xu, Yian Wang*, Qiushi Sun, Chengyou Jia, Kanzhi Cheng, Zichen Ding, Liheng Chen, Paul Pu Liang, Yu Qiao.arXiv 2024AgentStore: Scalable Integration of Heterogeneous Agents As Specialized Generalist Computer Assistant, Chengyou Jia, Minnan Luo, Zhuohang Dang, Qiushi Sun, Fangzhi Xu, Junlin Hu, Tianbao Xie, Zhiyong Wu.arXiv 2024Interative Evolution: A Neural-symbolic Self-Training Framework for Large Language Models, Fangzhi Xu, Qiushi Sun, Kanzhi Cheng, Jun Liu, Yu Qiao, Zhiyong Wu.LLMAgents@ICLR 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvement Zhiyong Wu, Chengcheng Han, Zichen Ding, Zhenmin Weng, Zhoumianze Liu, Shunyu Yao, Tao Yu, Lingpeng Kong.ACL 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents, Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, Zhiyong Wu.ACL 2024Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Models, Fangzhi Xu, Zhiyong Wu, et al.arXiv 2024TDAG: A Multi-Agent Framework based on Dynamic Task Decomposition and Agent Generation Yaoxiang Wang, Zhiyong Wu, Junfeng Yao, Jinsong SuCOLM 2024Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaboration Qiushi Sun, Zhangyue Yin, Xiang Li, Zhiyong Wu, Xipeng Qiu, Lingpeng Kong.
❓ In-Context Learning
arXiv 2023A Survey on In-context Learning, Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, et al.ACL 2024 (findings)LLM as Prompter: Low-resource Inductive Reasoning on Arbitrary Knowledge Graphs, Kai Wang, Yuwei Xu, Zhiyong Wu, Siqiang LuoACL 2023Self-adaptive In-context Learning, Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, Lingpeng Kong.ACL 2023 (demo)OpenICL: An Open-Source Framework for In-context Learning, Zhenyu Wu, YaoXiang Wang, Zhiyong Wu et al.ICML 2023Compositional Exemplars for In-context Learning, Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu, Lingpeng Kong.EMNLP 2023Can We Edit Factual Knowledge by In-Context Learning? Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, Baobao Chang
📃 Data Augmentation using LLMs
ICLR 2023Self-Guided High-Quality Data Generation in Efficient Zero-Shot Learning Jiahui Gao, Renjie Pi, Yong Lin, Hang Xu, Jiacheng Ye, Zhiyong Wu, et al.EMNLP 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generation, Jiacheng Ye, Jiahui Gao, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu and Lingpeng Kong.EMNLP 2022ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback, Jiacheng Ye, Jiahui Gao, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong.
🎼 Interpretability
ACL 2024 (findings)How Vocabulary Sharing Facilitates Multilingualism in LLaMA?, Fei Yuan, Shuai Yuan, Zhiyong Wu, Lei LiACL 2023 (findings)Explanation Regeneration via Information Bottleneck Qintong Li, Zhiyong Wu, Lingpeng Kong, Wei Bi.AAAI 2023Unsupervised Explanation Generation via Correct Instantiations Sijie Chen, Zhiyong Wu, Jiangjie Chen, Zhixing Li, Yang Liu, and Lingpeng KongACL 2021Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation, Zhiyong Wu, Lingpeng Kong, Wei Bi, Xiang Li, Ben Kao.ACL 2020Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT, Zhiyong Wu, Yun Chen, et al.WSDM 2020PERQ: Predicting, Explaining, and Rectifying Failed Questions in KB-QA Systems Zhiyong Wu, Ben Kao, Tien-Hsuan Wu, Pengcheng Yin, Qun Liu.
🧑🎨 Generative Model
arXiv 2024A Controlled Study on Long-Context Extension and Generalization in LLMs., Yi Lu, Jing Nathan Yan, Songlin Yang, Justin T. Chiu, Siyu Ren, Fei Yuan, Wenting Zhao, Zhiyong Wu, Alexander M. Rush.ICLR 2024EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modeling, Siyu Ren, Zhiyong Wu, Kenny Q Zhu.EMNLP 2023 (findings)DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion ModelsShansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, Lingpeng Kong.ICLR 2023DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, Lingpeng Kong.ACL 2021Lexical Knowledge Internalization for Neural Conversational Models Zhiyong Wu, Wei Bi, Xiang Li, Lingpeng Kong, Ben Kao.ACL 2021Cascaded Head-colliding Attention Lin Zheng, Zhiyong Wu, Lingpeng Kong